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Amendments to the Claims: 

This listing of claims will replace all prior versions, and listings, of claims in the application: 
Listing of Claims: 

1 . (Currently Amended) A hyperlinked broadcast system comprising: 

a video source providing video information for a video program including a plurality of 
consecutive video frames; 

an annotation system tracing movement of two or more video objects appearing in each 
of the plurality of consecutive video frames from a first location to a second location and 
generating annotation data to be associat e d with said vid e o information and generating 
annotation data timing information, wherein the g e nerating of the annotation data includes 
generating a single including only one mask for each video frame of the plurality of consecutive 
video frames, each mask including a plurality of pixels, wherein each of the plurality of pixels is 
mapp e d to an indicia for identifying a region or video object appearing in the corresponding 
vid e o frame, wher e in each pixel associated with the sam e r e gion or video object is mapped to the 
same indicia being stamped with the corresponding annotation data timing information based on 
a frame time of the corresponding video frame , each mask further including location and 
graphics data for at least on e of a plurality of of two or more graphics images to be overlaid on 
the two or more video objects appearing in the corresponding video frame; and 

an augmented video information transmission generator receiving said annotation data, 
said video information, and said annotation data timing information, said augmented video 
information transmission generator generating an augmented video transmission signal 
comprising said annotation data, said annotation data timing information, and said video 
information, and transmitting the augmented video transmission signal to a receiver, 

wherein said augmented video information transmission generator associates said video 
information with said annotation data using said annotation data timing information^.^ 
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wherein the receiver receiving the augmented video transmission signal is programmed, 
for each of the plurality of consecutive video frames, to: 

compare a current annotation data timing information with a current frame time of 
a current video frame; 

retrieve the location and graphics data of the two or more graphics images from 
the identified mask if the comparison results in a match; and 

overlay the two or more graphics images on the corresponding two or more video 
objects appearing in the video frame based on the retrieved location and graphics data, 

wherein the overlaying of the two or more graphics images is synchronized on a frame- 
by-frame basis with the movement of the two or more video objects from the first location to the 
second location over the plurality of consecutive video frames. 

2. (Original) The system of claim 1 wherein said augmented video information 
transmission generator comprises a vertical blanking interval insertion device. 

3. (Original) The system of claim 1 wherein said augmented video information 
transmission generator comprises at least one of a vertical ancillary data insertion device and a 
digital video data multiplexer. 

4. (Original) The system of claim 1 wherein said annotation data timing 
information comprises at least one of timestamp information, timecode information, frame 
numbering information, global time of day information, annotation data device commands, and a 
video program identifier. 

5. (Original) The system of claim 1 wherein said video information comprises 
digital video data. 
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6. (Original) The system of claim 1 wherein said video information comprises 
an analog video signal. 

7. (Original) The system of claim 1 further comprising: 
a post production environment; and 

a headend comprising said augmented video information transmission generator, 
wherein said video information and said annotation data timing information are combined 
by said post production environment and transmitted to said headend. 

8. (Original) The system of claim 7 wherein said headed is a cable headend. 

9. (Original) The system of claim 7 wherein said headend is a satellite headend. 
10-13. (Canceled) 

14. (Currently Amended) The system of claim [[13]]I wherein the receiver 

said display device displays said annotation data in response to a viewer request. 

15-16. (Canceled) 

17. (Currently Amended) The system of claim 1 wherein at least one of the 
mask[[s]] comprises location information of ffanll the two or more objects in the corresponding 
video frame. 

18. (Currently Amended) The system of claim 17 wherein said location 
information includes a graphics location reference that represents a fixed relation to a set of 
pixels associated with each said object. 
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19. (Original) The system of claim 18 wherein said graphics location reference 
includes an upper left most pixel in said associated pixel set. 

20. (Original) The system of claim 18 wherein said graphics location reference 
includes a centroid pixel of said associated pixel set. 

21. (Currently Amended) The system of claim 1 wherein at least one of the mask[[s]] 
comprises location and shape information of mthc two or more objects in the corresponding 
video frame. 

22-34. (Canceled) 

35. (Currently Amended) A method of generating a hyperlinked video signal 
comprising: 

receiving g en e rating annotation data timing information from video information for a 
video program including a plurality of consecutive video frames; 

tracing movement of two or more video objects appearing in each of the plurality of 
consecutive video frames from a first location to a second location and g enerating annotation 
data and annotation data timing information, for said video information , wherein the generating 
e£-the annotation data includes generating a single including only one mask for each video frame 
of the plurality of consecutive video frames, each mask including a plurality of pixels, wh e rein 
e ach of the plurality of pixels is mapp e d to an indicia for identifying a region or video object 
appearing in th e corr e sponding vid e o frame, wherein each pixel associated with th e sam e region 
or video obj e ct is mapped to th e same indicia being stamped with the corresponding annotation 
data timing information based on a frame time of the corresponding video frame , each mask 
further including location and g raphics data for at least one of a plurality of of two or more 
graphics images to be overlaid on the two or more video objects appearing in the corresponding 
video frame; 
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communicating said annotation data timing information, said annotation data, and said 
video information to an augmented video information transmission generator;-and 

synchronizing said video information with said annotation data in response to said 
annotation data timing information by said augmented video information transmission generator 
and generating an augmented video transmission signal comprising said annotation data, said 
annotation data timing information, and said video information; IT.11 

transmitting the augmented video transmission signal to a receiver, wherein the receiver 
receiving the augmented video transmission signal is programmed, for each of the plurality of 
consecutive video frames, to: 

compare a current annotation data timing information with a current frame time of 
a current video frame; 

retrieve the location and graphics data of the two or more graphics images from 
the identified mask if the comparison results in a match; and 

overlay the two or more graphics images on the corresponding two or more video 
objects appearing in the video frame based on the retrieved location and graphics data, 

wherein the overlaying of the two or more graphics images is synchronized on a frame- 
by-frame basis with the movement of the two or more video objects from the first location to the 
second location over the plurality of consecutive video frames. 

36. (Original) The method of claim 35 wherein said augmented video 
information transmission generator comprises a vertical blanking interval insertion device. 

37. (Original) The method of claim 35 wherein said augmented video 
information transmission generator comprises at least are of a vertical ancillary data insertion 
device and a digital video data multiplexer. 

38. (Original) The method of claim 35 wherein said annotation data timing 
information comprises at least one of timestamp information, timecode information, frame 
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numbering information, global time of day information, annotation data device commands, and a 
video program identifier. 

39. (Original) The method of claim 35 wherein said video information comprises 
digital video data. 

40. (Original) The method of claim 35 wherein said video information comprises 
an analog video signal. 

41. (Original) The method of claim 35 further comprising inserting said 
annotation data timing information in a vertical blanking interval of an analog video signal. 

42. (Original) The method claim 35 further comprising inserting said annotation 
data timing information in a vertical ancillary data region of a digital video signal. 

43. (Original) The method of claim 35 wherein said communicating step 
comprises transmitting said timing information and said video information to a broadcast 
network and subsequently to said augmented video information transmission generator. 

44-45. (Canceled) 

46. (Currently Amended) The method of claim 35 wherein at least one of the 
mask[[s]] comprises location information of the two or more a n-objects in the corresponding 
video frame. 

47. (Currently Amended) The method of claim 46 wherein said location information 
includes a graphics location reference that represents a fixed relation to a set of pixels associated 
with each said object. 
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48. (Original) The method of claim 47 wherein said graphics location reference 
includes an upper left most pixel of said associated pixel set. 

49. (Original) The method of claim 48 wherein said graphics location reference 
includes a centroid pixel of said associated pixel set. 

50. (Currently Amended) The method of claim [[44]]35 wherein at least one of the 
mask[[s]] comprises location and shape information of the two or more a n-objects in the 
corresponding video frame. 

51. (Currently Amended) The method of claim 50 wherein said shape information is 
represented by a graphical overlay of each said object. 

52. (Currently Amended) The method of claim 50 wherein said shape information is 
represented by an outline of each said object. 

53-54. (Canceled) 

55. (Previously Presented) The system of claim 1, wherein the annotation data 
further includes a plurality of object data packets, the mask further including an identifier to an 
object mapping table included in one or more of the plurality of object data packets, the object 
mapping table including at least one entry with an indicia from the corresponding mask 
identifying a particular video object, the entry further associating the indicia to information data 
structures included in one or more of the plurality of object data packets, the information data 
structures including information for the particular video object. 
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56. (Currently Amended) The system of claim 55 wherein the further including a 
television receiv e r, th e television receiver is further programmed being configured to: 

overlay a graphics image on a particular video frame for the particular video object bas e d 

on th e graphics data included in the corresponding mask; 

retrieve the identifier of the object mapping table from the corresponding mask 
responsive to a user selection associated with the overlaid graphics image; 

retrieve the object mapping table based on the retrieved identifier; 

identify the indicia in the corresponding mask for the particular video object for which 
the graphics image was overlaid; 

locate the entry in the object mapping table with the identified indicia; 
identify the information data structures associated with the located entry; 
retrieve the information in the identified information data structures; and 
display the retrieved information on the display device. 

57-60. (Canceled) 

61. (New) The system of claim 1, wherein the overlaying of the two or more graphics 
images is for alerting a viewer of interactive data associated with the two or more video objects 
prior to the viewer transmitting an interactive command with respect to one of the two or more 
video objects. 

62. (New) The method of claim 35, wherein the overlaying of the two or more 
graphics images is for alerting a viewer of interactive data associated with the two or more video 
objects prior to the viewer transmitting an interactive command with respect to one of the two or 
more video objects. 
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